Method and apparatus for intra prediction within display screen

ABSTRACT

The present invention relates to a method and apparatus for intra prediction. The intra prediction method for a decoder, according to the present invention, comprises the steps of entropy-decoding a received bitstream, generating reference pixels to be used in the intra prediction of a prediction unit; generating a prediction block from the reference pixels on the basis of a prediction mode for the prediction unit and reconstructing an image from the prediction block and a residual block, which is obtained as a result of entropy encoding, wherein the reference pixels and/or the prediction block pixels are predicted on the basis of a base pixel, and the predicted pixel value can be the sum of the pixel value of the base pixel and the difference between the pixel values of the base pixel and the generated pixel.

CROSS REFERENCE TO RELATED APPLICATIONS

The application is a Divisional of U.S. patent application Ser. No.14/606,008 (filed on Jan. 26, 2015), now issued as U.S. Pat. No.9,432,695, which is a Divisional of U.S. patent application Ser. No.14/118,973 (filed on Nov. 20, 2013), now issued as U.S. Pat. No.9,154,803, which is a National Stage Patent Application of PCTInternational Patent Application No. PCT/KR2012/003744 (filed on May 14,2012) under 35 U.S.C. §371, which claims priority to Korean PatentApplication Nos. 10-2011-0048130 (filed on May 20, 2011) and10-2011-0065210 (filed on Jun. 30, 2011), the teachings of which areincorporated herein in their entireties by reference.

BACKGROUND

The present invention relates to a video processing technique, and moreparticularly, to an intra prediction method in encoding/decoding videoinformation.

Recently, demands for high-resolution and high-quality images haveincreased in various fields of applications. As images have higherresolution and higher quality, amount of information on the images alsoincreases. Accordingly, when video data is transferred using media suchas existing wired and wireless broad band lines or is stored inconventional storage media, transfer and storage costs of the video datarise.

Thus, high-efficiency video compression techniques may be utilized toeffectively transmit, store or reproduce images with superior resolutionand superior quality.

SUMMARY

An aspect of the present invention is to provide a method of performingeffective intra prediction on a texture with directionality inconsideration of variations of reference pixels of neighboring blocks.

Another aspect of the present invention is to provide a method ofperforming planar prediction considering variations in pixel values ofadjacent blocks to a prediction block in conducting intra prediction.

Still another aspect of the present invention is to provide a method ofgenerating a reference pixel based on an intra-mode neighboring block ina position of an inter prediction mode neighboring pixel and using thereference pixel for intra prediction when constrained intra prediction(CIP) is employed.

Yet another aspect of the present invention is to provide a method ofgenerating a reference pixel considering variations in pixel value whenthe reference pixel is generated based on an intra-mode neighboringblock in a position of an inter prediction mode neighboring pixel.

An embodiment of the present invention provides an intra predictionmethod of an encoder, the method including generating reference pixelsfor intra prediction with respect to an input prediction unit,determining an intra mode for the prediction unit, generating aprediction block based on the reference pixels and the intra mode, andgenerating a residual block for the prediction unit and the predictionblock, wherein at least one of the reference pixels and pixels of theprediction block is predicted based on a base pixel, and a pixel valueof the predicted pixel is a sum of a pixel value of the base pixel and avariation in pixel value from the base pixel to the generated pixel.

A reference pixel of a neighboring block disposed on a top left cornerof the prediction block may be set as a first base pixel, a valueobtained by applying a variation in pixel value from the first basepixel to a lowest pixel among reference pixels of a neighboring blockdisposed on a left boundary of the prediction block and a variation inpixel value from the first base pixel to a rightmost pixel amongreference pixels of a neighboring block disposed on an upper boundary ofthe prediction block to the base pixel may be set as a pixel value of asecond base pixel as a diagonal pixel in a bottom right corner of theprediction block, and pixel values of diagonal pixels of the predictionblock may be predicted from the first base pixel and the second basepixel.

Here, non-diagonal pixels of the prediction block are predicted byinterpolation or extrapolation using the diagonal pixels and the pixelsof the neighboring blocks on the upper and/or left boundaries of theprediction block.

Further, a reference pixel of a neighboring block disposed on a top leftcorner of the prediction block may be set as the base pixel, and a valueobtained by applying a variation in pixel value from the base pixel to aneighboring pixel disposed in the same row as a prediction target pixelamong reference pixels of a neighboring block disposed on a leftboundary of the prediction block and a variation in pixel value from thebase pixel to a neighboring pixel disposed in the same column as theprediction target pixel among reference pixels of a neighboring blockdisposed on an upper boundary of the prediction block to the base pixelmay be predicted as a pixel value of the prediction target pixel.

Also, a pixel disposed in the same row or column as a prediction targetpixel among pixels of neighboring blocks disposed on left or upperboundary of the prediction block may be set as the base pixel, and avalue obtained by applying a variation in pixel value from the basepixel to the prediction pixel to the base pixel may be predicted as apixel value of the prediction target pixel.

Here, the prediction target pixel may be a diagonal pixel of theprediction block, and a non-diagonal pixel of the prediction block maybe predicted by interpolation using the diagonal pixel and the pixels ofthe neighboring blocks.

The intra prediction method may further include generating a referencepixel disposed on a boundary between an inter-mode block and theprediction unit when a block neighboring to the prediction unit is theinter-mode block, wherein a pixel disposed on a boundary of theprediction unit among pixels of an intra-mode block disposed on a leftside or lower side of the reference pixel may be set as a first basepixel, a pixel disposed on the boundary of the prediction unit amongpixels of an intra-mode block disposed on a right side or upper side ofthe reference pixel may be set as a second base pixel, and the referencepixel may be generated based on a distance from the first base pixel tothe reference pixel and a distance from the second base pixel to thereference pixel.

Here, a pixel value of the first base pixel may be an average pixelvalue of pixels disposed on the boundary of the prediction unit amongthe pixels of the intra-mode block that the first base pixel belongs to,and a pixel value of the second base pixel may be an average pixel valueof pixels disposed on the boundary of the prediction unit among thepixels of the intra-mode block that the second base reference belongsto. Further, a pixel value of the first base pixel may be a pixel valueof the reference pixel when an intra-mode block is disposed only on theleft side or lower side of the reference pixel, and a pixel value of thesecond base pixel may be a pixel value of the reference pixel when anintra-mode block is disposed only on the right side or upper side of thereference pixel.

Another embodiment of the present invention provides an intra predictionmethod of a decoder, the method including entropy-decoding a receivedbitstream, generating a reference pixel used for intra prediction of aprediction unit, generating a prediction block from the reference pixelbased on a prediction mode for the prediction unit, and reconstructing apicture from a residual block obtained by the entropy-decoding and theprediction block, wherein at least one of the reference pixels andpixels of the prediction block is predicted based on a base pixel, and apixel value of the predicted pixel is a sum of a pixel value of the basepixel and a variation in pixel value from the base pixel to thegenerated pixel.

A reference pixel of a neighboring block disposed on a top left cornerof the prediction block may be set as a first base pixel, a valueobtained by applying a variation in pixel value from the first basepixel to a lowest pixel among reference pixels of a neighboring blockdisposed on a left boundary of the prediction block and a variation inpixel value from the first base pixel to a rightmost pixel amongreference pixels of a neighboring block disposed on an upper boundary ofthe prediction block to the base pixel may be set as a pixel value of asecond base pixel as a diagonal pixel in a bottom right corner of theprediction block, and pixel values of diagonal pixels of the predictionblock may be predicted from the first base pixel and the second basepixel.

Here, non-diagonal pixels of the prediction block may be predicted byinterpolation or extrapolation using the diagonal pixels and the pixelsof the neighboring blocks on the upper and/or left boundaries of theprediction block.

A reference pixel of a neighboring block disposed on a top left cornerof the prediction block may be set as the base pixel, and a valueobtained by applying a variation in pixel value from the base pixel to aneighboring pixel disposed in the same row as a prediction target pixelamong reference pixels of a neighboring block disposed on a leftboundary of the prediction block and a variation in pixel value from thebase pixel to a neighboring pixel disposed in the same column as theprediction target pixel among reference pixels of a neighboring blockdisposed on an upper boundary of the prediction block to the base pixelmay be predicted as a pixel value of the prediction target pixel.

Further, a pixel disposed in the same row or column as a predictiontarget pixel among pixels of neighboring blocks disposed on a left orupper boundary of the prediction block may be set as the base pixel, anda value obtained by applying a variation in pixel value from the basepixel to the prediction pixel to the base pixel may be predicted as apixel value of the prediction target pixel.

Here, the prediction target pixel may be a diagonal pixel of theprediction block, and a non-diagonal pixel of the prediction block maybe predicted by interpolation using the diagonal pixel and the pixels ofthe neighboring blocks.

The intra prediction may further include generating a reference pixeldisposed on a boundary between an inter-mode block and the predictionunit when a block neighboring to the prediction unit is the inter-modeblock, wherein a pixel disposed on a boundary of the prediction unitamong pixels of an intra-mode block disposed on a left side or lowerside of the reference pixel may be set as a first base pixel, a pixeldisposed on the boundary of the prediction unit among pixels of anintra-mode block disposed on a right side or upper side of the referencepixel may be set as a second base pixel, and the reference pixel may begenerated based on a distance from the first base pixel to the referencepixel and a distance from the second base pixel to the reference pixel.

Here, a pixel value of the first base pixel may be an average pixelvalue of pixels disposed on the boundary of the prediction unit amongthe pixels of the intra-mode block that the first base pixel belongs to,and a pixel value of the second base pixel may be an average pixel valueof pixels disposed on the boundary of the prediction unit among thepixels of the intra-mode block that the second base reference belongsto. Further, a pixel value of the first base pixel may be a pixel valueof the reference pixel when an intra-mode block is disposed only on theleft side or lower side of the reference pixel, and a pixel value of thesecond base pixel may be a pixel value of the reference pixel when anintra-mode block is disposed only on the right side or upper side of thereference pixel.

The decoder may acquire an instruction to generate the pixels of theprediction block based on the base pixel through the entropy-decoding.Also, the decoder may acquire an instruction to generate the referencepixels based on the base pixel through the entropy-decoding.

As described above, according to the present invention, intra predictionon a texture with directionality may be achieved effectively inconsideration of variations of reference pixels of neighboring blocks.

Further, planar prediction may be performed considering variations inpixel values of neighboring blocks to a prediction block, therebyenhancing prediction efficiency.

In addition, when constrained intra prediction (CIP) is employed, areference pixel is generated based on an intra-mode neighboring block ina position of an inter-mode neighboring pixel and used for intraprediction, considering variations in pixel value, thereby enhancingprediction efficiency.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a configuration of a videoencoder according to an exemplary embodiment of the present invention.

FIG. 2 is a block diagram schematically illustrating a configuration ofan intra prediction module according to an exemplary embodiment of thepresent invention.

FIG. 3 is a block diagram illustrating a configuration of a videodecoder according to an exemplary embodiment of the present invention.

FIG. 4 schematically illustrates a planar prediction method.

FIG. 5 schematically illustrates an alternative planar predictionmethod.

FIG. 6 schematically illustrates that a diagonal pixel of a currentprediction block is predicted first.

FIG. 7 schematically illustrates a method of deriving other pixel valuesin the prediction block based on the diagonal pixel.

FIG. 8 schematically illustrates a method of predicting a pixel valueconsidering a reference pixel value and a variation from a referencepixel.

FIG. 9 schematically illustrates a method of deriving diagonal pixels ofa prediction block first and then pixel values of remaining pixels.

FIG. 10 schematically illustrates that diagonal pixels are derived firstand other pixels than the diagonal pixels are derived in the same methodas used for the diagonal pixels.

FIG. 11 schematically illustrates an CIP method.

FIG. 12 schematically illustrates an alternative CIP method.

FIG. 13 schematically illustrates that a system according to the presentinvention conducts CIP in consideration of variations in pixel value.

FIG. 14 is a flowchart schematically illustrating an operation of theencoder in the system according to the present invention.

FIG. 15 illustrates a prediction direction of an intra prediction mode.

FIG. 16 is a flowchart schematically illustrating an operation of thedecoder in the system according to the present invention.

DETAILED DESCRIPTION OF EMBODIMENTS

Although elements shown in the drawings are independently shown so as todescribe different features and functions of a video encoder/decoder,such a configuration does not indicate that each element is constructedby a separate hardware constituent or software constituent. That is, theelements are independently arranged and at least two elements may becombined into a single element, or a single element may be divided intoa plurality of elements to perform functions. It is to be noted thatembodiments in which some elements are integrated into one combinedelement and/or an element is divided into multiple separate elements areincluded in the scope of the present invention without departing fromthe essence of the present invention.

Hereinafter, exemplary embodiments of the present invention will bedescribed in detail with reference to the accompanying drawings. Likereference numerals in the drawings refer to like elements throughout,and redundant descriptions of like elements will be omitted herein.

FIG. 1 is a block diagram illustrating a configuration of a videoencoder according to an exemplary embodiment of the present invention.Referring to FIG. 1, the video encoder includes a picture partitionmodule 110, an inter prediction module 120, an intra prediction module125, a transform module 130, a quantization module 135, a dequantizationmodule 140, an inverse transform module 145, a deblocking filter 150, amemory 160, a rearrangement module 165 and an entropy encoding module170.

The picture partition module 110 may receive input of a current pictureand divide the picture into at least one coding unit. A coding unit is aunit of encoding conducted by the video encoder and may be also referredto as a CU. A coding unit may be recursively subdivided with a depthbased on a quadtree structure. A coding unit having a maximum size isreferred to as a largest coding unit (LCU), and a coding unit with aminimum size as a smallest coding unit (SCU). A coding unit may have asize of 8×8, 16×16, 32×32 or 64×64. The picture partition module 110 maypartition or devide the coding unit to generate a prediction unit and atransform unit. The prediction unit may be also referred to as a PU, andthe transform unit may be also referred to as a TU.

In an inter prediction mode, the inter prediction module 120 may performmotion estimation (ME) and motion compensation (MC). The interprediction module 120 generates a prediction block based on informationon at least one of previous and subsequent pictures of the currentpicture, which may be referred to as inter-frame prediction.

The inter prediction module 120 is provided with a partitionedprediction target block and at least one reference block stored in thememory 160. The inter prediction module 120 performs motion estimationusing the prediction target block and the reference block. The interprediction module 120 generates motion information including a motionvector (MV), a reference block index and a prediction mode as a resultof motion estimation.

Further, the inter prediction module 120 performs motion compensationusing the motion information and the reference block. Here, the interprediction module 120 generates and outputs a prediction blockcorresponding to an input block from the reference block.

The motion information is entropy-encoded to form a compressedbitstream, which is transmitted from the video encoder to a videodecoder.

In an intra prediction mode, the intra prediction module 125 maygenerate a prediction block based on information on a pixel in thecurrent picture. Intra prediction is also referred to as intra-frameprediction. In the intra prediction mode, a prediction target block anda reconstructed block reconstructed through encoding and decoding areinput to the intra prediction module 125. Here, the reconstructed blockis a picture that has not been subjected to the deblocking filter. Thereconstructed block may be a previous prediction block.

FIG. 2 is a block diagram schematically illustrating a configuration ofthe intra prediction module according to an exemplary embodiment of thepresent invention. Referring to FIG. 2, the intra prediction moduleincludes a reference pixel generation module 210, an intra predictionmode determination module 220, and a prediction block generation module230.

The reference pixel generation module 210 generates a reference pixelneeded for intra prediction. Pixels in a rightmost vertical line of aleft block neighboring to a prediction target block and pixels in alowest horizontal line of an upper block neighboring to a predictiontarget block are used for generating the reference pixel. For example,when the prediction target block has a size of N, 2N pixels in each ofleft and upper directions are used as reference pixels. The referencepixel may be used as it is or via adaptive intra smoothing (AIS)filtering. When the reference pixel is subjected to MS filtering,information about AIS filtering is signaled.

The intra prediction mode determination module 220 receives input of theprediction target block and the reconstructed block. The intraprediction mode determination module 220 selects a mode minimizingquantity of information to be encoded among prediction modes using theinput picture and outputs information on the prediction mode. Here, apreset cost function or Hadamard transform may be used.

The prediction block generation module 230 receives input of theinformation on the prediction mode and the reference pixel. Theprediction block generation module 230 spatially predicts andcompensates a pixel value of the prediction target block using theinformation on the prediction mode and a pixel value of the referencepixel, thereby generating a prediction block.

The information on the prediction mode is entropy-encoded to form acompressed bitstream along with video data and transmitted from thevideo encoder to the video decoder. The video decoder uses theinformation on the prediction mode when generating an intra predictionblock.

Referring back to FIG. 1, a differential block is generated bydifference between the prediction target block and the prediction blockgenerated in the inter or intra prediction mode and is input to thetransform module 130. The transform module 130 transforms thedifferential block in a transform unit to generate a transformcoefficient.

A transform block with a transform unit has a quadtree structure withinmaximum and minimum sizes and thus is not limited to a predeterminedsize. Each transform block has a flag indicating whether the currentblock is partitioned into sub-blocks, wherein when the flag is 1, thecurrent transform block may be split into four sub-blocks. Discretecosine transform (DCT) may be used for transformation.

The quantization module 135 may quantize the values transformed by thetransform module 130. A quantization coefficient may change based on ablock or importance of a picture. The quantized transform coefficientmay be provided to the rearrangement module 165 and the dequantizationmodule 140.

The rearrangement module 165 may change a two-dimensional (2D) block ofthe transform coefficients into a one-dimensional (1D) vector oftransform coefficients by scanning so as to enhance efficiency inentropy encoding. The rearrangement module 165 may change scanning orderbased on stochastic statistics to enhance entropy encoding efficiency.

The entropy encoding module 170 entropy-encodes the values obtained bythe rearrangement module 165, and the encoded values are formed into acompressed bitstream, which is stored or transmitted through a networkabstraction layer (NAL).

The dequantization module 140 receives and dequantizes the transformcoefficients quantized by the quantization module 135, and the inversetransform module 145 inverse-transforms the transform coefficients,thereby generating a reconstructed differential block. The reconstructeddifferential block is merged with the prediction block generated by theinter prediction module 120 or the intra prediction module 125 togenerate a reconstructed block. The reconstructed block is provided tothe intra prediction module 125 and the deblocking filter 150.

The deblocking filter 150 filters the reconstructed block to remove adistortion on a boundary between blocks that occurs in encoding anddecoding processes and provides a filtered result to an adaptive loopfilter (ALF) 155.

The ALF 155 performs filtering to minimize an error between theprediction target block and the final reconstructed block. The AFL 155performs filtering based on a value resulting from comparison of thereconstructed block filtered by the deblocking filter 150 and thecurrent prediction target block, and a filter coefficient information onthe ALF 155 is loaded into a slice header and transmitted from theencoder to the decoder.

The memory 160 may store the final reconstructed block obtained throughthe ALF 155, and the (final) stored reconstructed block may be providedto the inter prediction module 120 to perform inter prediction.

FIG. 3 is a block diagram illustrating a configuration of a videodecoder according to an exemplary embodiment of the present invention.Referring to FIG. 3, the video decoder includes an entropy decodingmodule 310, a rearrangement module 315, a dequantization module 320, aninverse transform module 325, an inter prediction module 330, an intraprediction module 335, a deblocking filter 340, an ALF 345, and a memory350.

The entropy decoding module 310 receives a compressed bitstream from anNAL. The entropy decoding module 310 entropy-decodes the receivedbitstream, and also entropy-decodes a prediction mode and motion vectorinformation if the bitstream includes the prediction mode and the motionvector information. An entropy-decoded transform coefficient ordifferential signal is provided to the rearrangement module 315. Therearrangement module 315 inverse-scans the transform coefficient ordifferential signal to generate a 2D block of transform coefficients.

The dequantization module 320 receives and dequantizes theentropy-decoded and rearranged transform coefficients. The inversetransform module 325 inverse-transforms the dequantized transformcoefficients to generate a differential block.

The differential block may merge with a prediction block generated bythe inter prediction module 330 or intra prediction module 335 togenerate a reconstructed block. The reconstructed block is provided tothe intra prediction module 335 and the deblocking filter 340. The interprediction module 330 and the intra prediction module 335 may performthe same operations as the inter prediction module 120 and the intraprediction module 125 of the video encoder.

The deblocking filter 340 filters the reconstructed block to remove adistortion on a boundary between blocks that occurs in encoding anddecoding processes and provides a filtered result to the ALF 345. TheALF 345 performs filtering to minimize an error between the predictiontarget block and the finally reconstructed block. The memory 160 maystore the final reconstructed block obtained through the ALF 345, andthe stored (final) reconstructed block may be provided to the interprediction module 330 to perform inter prediction.

Meanwhile, in an area with insignificant changes in texture, forexample, a monotonous background of the sky or sea, planar intraprediction is used to further enhance encoding efficiency.

Intra prediction is classified into directional prediction, DCprediction and planar prediction, wherein planar prediction may be anextended concept of DC prediction. Although planar prediction may bebroadly included in DC prediction, planar prediction may cover aprediction method that DC prediction does not deal with. For example, DCprediction is preferable for a uniform texture, while planar predictionis effective for block prediction in a pixel values havingdirectionality.

The present specification illustrates a method of improving planarprediction efficiency with respect to a texture with directionalityusing variations in pixel values of reference pixels of neighboringblocks.

FIG. 4 schematically illustrates a planar prediction method.

Referring to FIG. 4(A), a pixel value 425 of a pixel in a bottom rightcorner of a current block 420 is predicted. The pixel value 425 of thepixel on the bottom right corner of the current block may be predictedas a DC value.

Referring to FIG. 4(B), pixel values of pixels located on a rightboundary of the current block and pixel values of pixels located on abottom boundary of the current block are predicted. For example, a pixelvalue 445 located on the right boundary of the current block may bepredicted by linear interpolation of a pixel value 450 of an upper blockand the DC value 425. Further, a pixel value 435 located on the bottomboundary of the current block may be predicted by linear interpolationof a pixel value 430 of a left block and the DC value 425.

Referring to FIG. 4(C), pixel values of remaining pixels, other than thepixel on the bottom right corner, the pixels on the right boundary andthe pixels on the bottom boundary in the current block, may be predictedby bilinear interpolation using the pixel values of the upper and leftblocks and the already predicted pixel values in the current block. Forexample, a pixel value 475 in the current block may be predicted byinterpolation using a pixel value 460 of the upper block, a pixel value455 of the left block, the already predicted pixel value 445 located onthe right boundary of the current block and the already predicted pixelvalue 435 located on the bottom boundary of the current block.

Referring to FIG. 4(D), the prediction samples (predicted samples)obtained via the foregoing process may be refined. For example, a pixelvalue X 485 in the current block may be refined using an upper samplevalue T 480 and a left sample value L 490. Specifically, X′ refined fromX may be obtained by X′={(X<<1)+L+T+1}>>2. Here, “x<<y” indicates that atwo's complement integer expression of x is arithmetically shifted tothe left by a binary unit y, while “x>>y” indicates that the two'scomplement integer expression of x is arithmetically shifted to theright by the binary unit y.

FIG. 5 schematically illustrates an alternative planar predictionmethod.

In the method of FIG. 5, pixel values of pixels diagonally located in acurrent pixel are predicted first, and pixel values of remaining pixelsin the current block are predicted using the predicted pixel values. Forconvenience of description, the pixels located diagonally from the topleft to the bottom right among the pixels constructing the block arereferred to as diagonal pixels below.

Referring to FIG. 5(A), pixel values of diagonal pixels 540 of a currentblock 510 are predicted using a pixel value 520 of an upper referenceblock and a pixel value 530 of a left reference block. For instance, apixel value of a diagonal pixel P in the current block may be obtainedusing a pixel value of a pixel AboveRef located on a boundary betweenthe current block and the upper block among pixels of the upper blockand a pixel value of a pixel LeftRef located on a boundary between thecurrent block and the left block among pixels of the left block byP=(LeftRef+AboveRef+1)>>1.

Referring to FIG. 5(B), pixel values of pixels in the current block 510other than the diagonal pixels 540 may be obtained by linearinterpolation using the pixel value obtained in FIG. 5(A) and pixelvalues of the pixels of the upper and left blocks on the boundaries. Forexample, P1 may be obtained using the pixel AboveRef of the upper blockand the obtained diagonal pixel P by P1=(AboveRef*d2+P*d1)/(d1+d2).Further, P2 may be obtained by P2=(LeftRef*d3+P*d4)(d3+d4).

Meanwhile, the planar prediction methods illustrated in FIGS. 4 and 5are effective for a uniform texture without directionality, whereasthese methods may have reduced efficiency in prediction in a case of atexture with directionality, such as luma pixels in which luminancechanges substantially in one direction, for example, a horizontaldirection, but hardly changes in another direction, for example, avertical direction.

Thus, planar intra prediction considering variations in pixel value maybe needed. Planar intra prediction according to the present inventionselects or predict a base pixel value and applies variations in pixelvalues between a base pixel and a target pixel to the base pixel value,thereby predicting a pixel value of the target pixel.

Hereinafter, examples of the present invention will be described withreference to the drawings.

Example 1

FIG. 6 schematically illustrates that a diagonal pixel Pii of a currentprediction block is predicted first. Although FIG. 6 illustrates an 8×8prediction block for convenience of description, the present inventionmay be also applied to an N×N prediction block, without being limited tothe 8×8 prediction block.

In Example 1 shown in FIG. 6, the diagonal pixels of the currentprediction block are predicted first based on a reference pixel (Ri0and/or R0j, 0≦i,j≦8 in 8×8 prediction block case) of a reference blockneighboring to the current prediction block.

That is, after the diagonal pixels Pii are obtained, other pixel valuesin the prediction block may be derived by interpolation or extrapolationusing reference pixel values (Rij) of the neighboring block and the Pii.

FIG. 7 schematically illustrates a method of deriving the other pixelvalues in the prediction block based on the diagonal pixels.

In the present invention, planar prediction is carried out inconsideration of changes in pixel values. For example, as shown in FIG.7(A), when the reference pixel values increase both in an x direction(right) and in a y direction (down), pixel values in the predictionblock are also more likely to increase in a bottom-right direction. Inthis case, a pixel value of P88 on a bottom right corner of theprediction block may be predicted first, and other pixels may bepredicted based on the pixel value of P88.

To predict the value of P88, defining a pixel value of reference pixelR00 on a top left corner of the current prediction block as a pixelvalue of base pixel, a variation from the base pixel to the predictiontarget pixel P88 in the prediction block may be applied to the pixelvalue of the base pixel. For example, a pixel value of the target pixelP88 may be obtained by Equation 1. For convenience of description, Rijor Pij illustrated in the drawings and the specification are presentedas R_(i,j) and P_(i,j).

P _(8,8) =R _(0,0)+√{square root over ((R _(0,8) −R _(0,1))²+(R _(8,0)−R _(1,0))²)}  [Equation 1]

When P88 is obtained, the other diagonal pixels Pii may be obtained byEquation 2.

$\begin{matrix}{P_{i,i} = {R_{0,0} + {\frac{i}{8} \cdot \left( {P_{8,8} - R_{0,0}} \right)}}} & \left\lbrack {{Equation}\mspace{14mu} 2} \right\rbrack\end{matrix}$

Here, since the present example illustrates the 8×8 prediction block, imay be 1, 2, . . . , 8. Although Example 1 illustrates the 8×8prediction block for convenience of description, in an N×N predictionblock, Pii may be obtained as Pii=R00+(i/N)P88.

As shown in FIG. 7(B), even when the reference pixel value decrease bothin the x direction (right) and in the y direction (down), a pixel valueof P88 in the bottom right corner of the prediction block may be derivedin consideration of variations in decreasing pixel values, and the otherpixel values may be predicted based on the pixel value of P88. In thiscase, P88 may be derived by Equation 3.

P _(8,8) =R _(0,0)−√{square root over ((R _(0,8) −R _(0,1))²+(R _(8,0)−R _(1,0))²)}  [Equation 3]

When P88 is obtained, the other diagonal pixels in the prediction blockmay be obtained by Equation 4.

$\begin{matrix}{P_{i,i} = {R_{0,0} + {\frac{i}{8} \cdot \left( {P_{8,8} - R_{0,0}} \right)}}} & \left\lbrack {{Equation}\mspace{14mu} 4} \right\rbrack\end{matrix}$

Here, i may be 1, 2, . . . , 8.

As shown in FIG. 7(C), when the reference pixel values increase in atop-right direction, the diagonal pixels located from the bottom left tothe top right in the prediction block are derived first based onvariations in pixel values, unlike in FIGS. 7(A) and 7(B). For example,a pixel value of P81 in a bottom left corner of the prediction block isderived, and the remaining pixel values may be predicted based on thepixel value of P81. In this case, P81 may be derived by Equation 5.

P _(8,1) =R _(0,9)−√{square root over ((R _(0,8) −R _(0,1))²+(R _(8,0)−R _(1,0))²)}  [Equation 5]

When P81 is obtained, the remaining diagonal pixels (bottom left to topleft) in the prediction block may be derived by Equation 6.

$\begin{matrix}{P_{i,{9 - i}} = {R_{0,9} + {\frac{i}{8} \cdot \left( {P_{8,1} - R_{0,9}} \right)}}} & \left\lbrack {{Equation}\mspace{14mu} 6} \right\rbrack\end{matrix}$

Here, i may be 1, 2, . . . , 8.

Also, as shown in FIG. 7(D), when the reference pixel values increase ina bottom-left direction, the diagonal pixels located from the bottomleft to the top right in the prediction block are derived first based onvariations in pixel values. For example, the pixel value of P81 in thebottom left corner of the prediction block is derived, and the remainingpixel values may be predicted based on the pixel value of P81. In thiscase, P81 may be derived by Equation 7.

P _(8,1) =R _(0,9)+√{square root over ((R _(0,8) −R _(0,1))²+(R _(8,0)−R _(1,0))²)}  [Equation 7]

When P81 is obtained, the remaining diagonal pixels (bottom left to topleft) in the prediction block may be derived by Equation 8.

$\begin{matrix}{P_{i,{9 - i}} = {R_{0,9} + {\frac{i}{8} \cdot \left( {P_{8,1} - R_{0,9}} \right)}}} & \left\lbrack {{Equation}\mspace{14mu} 8} \right\rbrack\end{matrix}$

Here, i may be 1, 2, . . . , 8.

In view of calculation loads, approximation of the calculations ofsquare roots to derive the diagonal pixels may be considered as inEquation 9.

Δx+Δy≈√{square root over (Δx² +Δy ²)}  [Equation 9]

Subsequently, the other pixel values in the prediction block may bederived by interpolation or extrapolation using the prediction values ofthe diagonal pixels, upper reference pixel values and left referencepixel values.

In FIGS. 7(A) and 7(B), the pixels Pij in the prediction block may bederived by interpolation using the diagonal pixels Pii and referencepixels R of the neighboring block. Here, an interpolation shown inEquation 10 may be used.

P _(i,j)=(R _(0,j) *d ₂ +P _(i,i) *d ₁)/(d ₁ +d ₂)

or

P _(i,j)=(R _(i,0) *d ₂ +P _(i,i) *d ₁)/(d ₁ +d ₂)  [Equation 10]

Here, d₁ is a distance from the pixel R0j or Rj0 of the neighboringblock used for interpolation to the prediction target pixel Pij, and d₂is a distance from the diagonal pixel Pii used for interpolation to theprediction target pixel Pij.

Further, in FIGS. 7(C) and 7(D), the pixel Pi derived by interpolationamong the pixels in the prediction block may be derived by Equation 11.

P _(i,j)=(R _(i,0) *d ₂ +P _(i,i) *d ₁)/(d ₁ +d ₂)  [Equation 11]

or

P _(i,j)=(R _(i,0) *d ₂ +P _(i,9-i) *d ₁)/(d ₁ +d ₂)

Here, i+j<9, and d₁ is a distance from the pixel R0j or Rj0 of theneighboring block used for interpolation to the prediction target pixelPij, and d₂ is a distance from the diagonal pixel Pii used forinterpolation to the prediction target pixel Pij. Here, althoughEquation 11 is used for interpolation to derive the pixel Pij of theprediction block, various interpolation methods may be employed in thepresent invention, without being limited thereto.

Meanwhile, in FIGS. 7(C) and 7(D), there is a pixel Pe derived byextrapolation among the pixels of the prediction block. Here, anextrapolation shown in Equation 12 may be used to derive the pixel inthe prediction block.

$\begin{matrix}{{P_{i,j} = {R_{i,0} + {\left( {P - R_{i,0}} \right)*\left( {1 + \frac{d_{2}}{d_{1}}} \right)\mspace{14mu} {or}}}}{P_{i,j} = {R_{0,j} + {\left( {P - R_{0,j}} \right)*\left( {1 + \frac{d_{2}}{d_{1}}} \right)}}}} & \left\lbrack {{Equation}\mspace{14mu} 12} \right\rbrack\end{matrix}$

In this case, i+j>9 and P is a diagonal pixel used for extrapolation.Further, as described above, d₁ and d₂ are a distance from the referencepixel to the prediction target pixel Pij and a distance from the pixelPii to the prediction target pixel Pij, respectively.

Example 2

FIG. 8 schematically illustrates another method of predicting a pixelvalue considering a base pixel value and a variation from a base pixel.Although FIG. 8 illustrates an 8×8 prediction block for convenience ofdescription, the present invention may be also applied to an N×Nprediction block, without being limited to the 8×8 prediction block.

FIG. 8 illustrates a reference pixel P00 located on the top left cornerof the prediction block as a base pixel. In Example 2, a predictiontarget pixel Pij is derived by applying vertical and horizontalvariations from the reference pixel to the base pixel value.

For example, the target pixel Pij is derived by Equation 13.

Pij=R00+Δx+Δy  [Equation 13]

Here, Δy=Ri0−R00, Δx=R0j−R00, and 1≦i, j≦8 in the 8×8 prediction blockcase.

For example, referring to FIG. 8, a pixel P33 is derived byP33=R00+Δx+Δy according to Equation 7. Here, Δx and Δy are variations inpixel value in the x direction and the y directions from the base pixelR00 to P33.

Alternatively, referring to FIG. 8, a pixel P76 is derived byP76=R00+Δx′+Δy′ according to Equation 13. Here, Δx′ and Δy′ arevariations in pixel value in the x direction and the y directions fromthe base pixel R00 to P76.

Example 3

FIG. 9 schematically illustrates another method of deriving diagonalpixels of a prediction block first and then pixel values of remainingpixels.

Although FIG. 5 illustrates that the diagonal pixels are derived basedon an average value of two pixels in a horizontal/vertical direction ofa neighboring block to the current prediction block, Example 3 shown inFIG. 9 drives the diagonal pixels in consideration of variations.

Referring to FIG. 9(A), the diagonal pixels of the prediction block arepredicted using pixel values of neighboring blocks located on upperand/or left boundaries of the prediction block. For example, thediagonal pixels Pii is predicted by Equation 14.

P _(i,i) =R _(0,i) +Δy

or

P _(i,i) =R _(i,0) +Δx  [Equation 14]

For example, referring to FIG. 9(A), P33 may be predicted by P33=R03+Δyor P33=R30+Δx according to Equation 14. Δx and Δy are variations inpixel value in the x direction from a base pixel R30 to P33 and in the ydirection from a base pixel R03 to P33, respectively.

Referring to FIG. 9(B), other pixels Pij of the current block than thediagonal pixels may be predicted by linear interpolation using theprediction values of the diagonal pixels and reference pixels R00, R10to R80 and R01 to R08 of the neighboring blocks on the upper and leftboundaries of the current block.

For example, a pixel value Pij may be predicted by Equation 15.

$\begin{matrix}{{P_{ij} = {\frac{{R\; 0\; j \times d\; 2} + {{Pii} \times d\; 1}}{{d\; 1} + {d\; 2}}\mspace{14mu} {or}}}{P_{ij} = \frac{{{Ri}\; 0 \times d\; 2} + {{Pii} \times d\; 1}}{{d\; 1} + {d\; 2}}}} & \left\lbrack {{Equation}\mspace{14mu} 15} \right\rbrack\end{matrix}$

d1 is a distance from the pixel R0j or Pi0 of the neighboring blocksused for interpolation to the prediction target pixel Pij, and d2 is adistance from the diagonal pixel Pii used for interpolation to theprediction target pixel Pij.

Example 4

FIG. 10 schematically illustrates that diagonal pixels are derived firstand other pixels than the diagonal pixels are derived in the same methodas used for the diagonal pixels.

In FIG. 10, the diagonal pixels may be predicted in the same manner asillustrated in FIG. 9. Thus, referring to FIG. 10(A), a diagonal pixelP33 of a current prediction block may be predicted by P33=R03+Δy orP33=R30+Δx.

Subsequently, other pixels Pij of the current block than the diagonalpixels may be predicted by linear interpolation using the predictionvalues of the diagonal pixels and reference pixels R00, R10 to R80 andR01 to R08 of neighboring blocks on upper and left boundaries of thecurrent block.

Here, the same method as used for deriving diagonal pixels may be used.For example, a pixel Pij may be predicted by Equation 16.

Pij=R0j+Δy

or

Pij=Ri0+Δx  [Equation 16]

Here, Δy=Ri0−R00, Δx=R0j−R00, and 1≦i, j≦8 in the 8×8 prediction blockcase.

For example, referring to FIG. 10, P37 may be derived by P37=R07+Δy orP37=R70+Δx according to Equation 16.

Meanwhile, accumulation of minor errors resulting from integerarithmetic by the encoder or decoder for a long time may cause a seriouserror. Further, when a transmission error occurs in a neighboring blockto a current block, a mismatch arises between the encoder and thedecoder or the error spread. For example, when an error occurs in theneighboring block, pixel values on a boundary of the neighboring blockare changed. In this case, when the decoder uses a pixel with a changedpixel value as a reference pixel, the error spreads to the currentblock. Thus, a tool to prevent such a problem is needed, for example, anencoding tool such as constrained intra prediction (CIP).

FIG. 11 schematically illustrates an CIP method.

In the method of FIG. 11, if there is any one inter prediction modeblock neighboring to a current macroblock T, a DC intra prediction modeis used only and a DC prediction value is fixed to 128.

Here, a pixel value of a block predicted by the inter prediction modeamong the neighboring blocks is not used as a reference pixel value.Thus, in this method, a DC prediction mode is compulsorily employedexcluding even available information, for example, neighboring intraprediction mode pixels.

FIG. 12 schematically illustrates an alternative CIP method.

In the method of FIG. 12, a pixel value of a block predicted in theintra prediction mode among neighboring blocks is used as a referencepixel value, and a pixel value of a block predicted in the interprediction mode is derived using neighboring intra prediction modeblocks. Thus, not only the DC mode but other intra prediction modes maybe used.

Referring to FIG. 12, among neighboring blocks to a current predictionblock T, pixel values 1210, 1220 and 1230 of blocks A, B, D, E, F, H andI predicted by the inter prediction mode are derived using pixels ofblocks predicted by the intra prediction mode.

For example, when predicted pixels of intra prediction mode are presenton both right and left sides of a target inter-prediction sample, apixel value P_(T) of a block predicted by the inter prediction mode isderived by Equation 17.

P _(T)=(P _(LB) +P _(RA)+1)>>1  [Equation 17]

Here, P_(T) is a target inter-prediction sample, P_(LB) is a left orlower intra-prediction sample, and P_(RA) is a right or upperintra-prediction sample. Further, when an intra-prediction sample ispresent on either side of the target inter-prediction sample only, apixel value P_(T) of a block predicted by the inter prediction mode isderived by Equation 18.

P _(T) =P _(RA) or P _(T) =P _(LB)  [Equation 18]

The method of FIG. 12 utilizes the intra prediction mode more properlythan the method of FIG. 11 but uses an average value of available intraprediction mode pixel values or an available intra prediction mode pixelvalue itself as a pixel value of a neighboring block predicted in theinter prediction mode, without considering variation in pixel values.

Thus, a CIP method considering variations in pixel value is needed.

Example 5

FIG. 13 schematically illustrates that a system according to the presentinvention performs CIP in consideration of variations in pixel value.

The method of FIG. 13 using variations in pixel values of both pixelsfor interpolation achieves more accurate prediction of a target pixelvalue than the method of FIG. 12 using an average value of both pixelvalues as a pixel value to derive. For example, a target pixel PT amongpixel values 1310, 1320 and 1330 to be derived may be derived byEquation 19.

$\begin{matrix}{P_{T} = \frac{{P_{LB} \times d\; 2} + {P_{RA} \times d\; 1}}{{d\; 1} + {d\; 2}}} & \left\lbrack {{Equation}\mspace{14mu} 19} \right\rbrack\end{matrix}$

Here, P_(T) is a target prediction sample, P_(LB) is a left or lowerintra-prediction sample, and P_(RA) is a right or upper intra-predictionsample. Further, as shown in FIG. 13, d1 is a distance from P_(LB) toP_(T), and d2 is a distance from P_(RA) to P_(T).

For instance, referring to FIG. 13, P_(T)1 may be derived by(P_(LB)1*d2₁+P_(RA)1*d1₁)/(d1₁+d2₁), and P_(T)2 may be derived by(P_(LB)2*d2₂+P_(RA)2*d1₂)/(d1₂+d2₂).

If an intra-prediction sample to be used for interpolation is present oneither right and left sides or either upper and lower sides of thetarget prediction sample P_(T) only, P_(T)=P_(LB) or P_(T)=P_(RA).Further, if there is no block predicted in the intra prediction modeneighboring to the target prediction block T, a pixel value in the sameposition as in a previous picture may be copied for use as a referencepixel value.

Average values of intra pixels on the boundary may be used as P_(LB) orP_(RA) value. For example, in FIG. 3, when P_(T) is located in a lowerpixel row 1320 of an E block or D block, an average value of four lowestpixels of an intra prediction mode C block may be used as P_(RA), and anaverage value of eight rightmost pixels of a G block may be used asP_(LB). In this case, a reference point of d1 is a top pixel among therightmost pixels of the G block, and a reference point of d2 is aleftmost pixel among the lowest pixels of the C block.

Further, linear interpolation gives smoothing effect on boundary pixelsand thus adaptive intra smoothing (AIS) may be off. Here, in the DCprediction mode, filtering on pixels on a boundary of the predictionblock may be on.

FIG. 14 is a flowchart schematically illustrating an operation of theencoder in the system according to the present invention.

Referring to FIG. 14, a new prediction unit of a current picture isinput (S1410). The prediction unit (PU) may be a basic unit for intraprediction and inter prediction. The prediction unit may be a smallerblock than a coding unit (CU) and may be a rectangular shape, notnecessarily a square shape. Intra prediction of the prediction unit isbasically carried out by a 2N×2N or N×N block.

Subsequently, a reference pixel needed for intra prediction is derived(S1420). Pixels in a rightmost vertical line of a left block neighboringto a current prediction block and pixels in a lowest horizontal line ofan upper block neighboring to the current prediction block are used forgenerating the reference pixel. When the prediction block has an N size,2N pixels of the left and upper blocks are used in total as referencepixels.

Here, the pixels in the rightmost vertical line of the left blockneighboring to the current prediction block and the pixels in the lowesthorizontal row of the upper block neighboring to the current predictionblock may be used as the reference pixels as they are or via smoothing.

When smoothing is involved, smoothing information may be also signaledto the decoder. For example, when smoothing is performed, an AIS filtermay be employed, in which filter coefficients [1, 2, 1] or [1, 1, 4, 1,1] may be used. Among these two coefficients, the latter filtercoefficient may provide a sharper boundary. As mentioned above,information including whether to use a filter, a type of a filter to useand a filter coefficient may be signaled to the decoder.

Meanwhile, when CIP is used for generating the reference pixel, aCIP_flag value is set to 1. When CIP is applied, only pixels ofneighboring blocks encoded in the intra prediction mode are used asreference pixels and pixels of the neighboring blocks encoded in theinter prediction mode are not used as reference pixels. In this case, asshown in FIG. 13, pixels (target prediction samples) corresponding topositions of the pixels of the neighboring blocks encoded in the interprediction mode are generated as reference pixels by interpolating theneighboring reference pixels encoded in the intra prediction mode, orthe neighboring reference pixels encoded in the intra prediction modeare copied and used as reference pixels corresponding to positions ofthe pixels of the neighboring blocks encoded in the inter predictionmode.

For example, when intra prediction mode prediction pixels are present onboth right and left sides, and upper and lower sides of a targetinter-prediction sample, the target prediction sample P_(T) located in ablock predicted in the inter prediction mode may be derived by Equation11. Further, when an intra-prediction sample is present only on eitherside of the target prediction sample, the target prediction sample P_(T)located in a block location predicted in the inter prediction mode maybe derived by Equation 12. In Equation 11 and/or Equation 12, averagevalues of the corresponding intra prediction mode pixels may be used asP_(LB) and P_(RA) values. If there is no neighboring block predicted inthe intra prediction mode, a pixel value in the same position as in aprevious picture may be copied for use as a reference pixel value.

Since linear interpolation gives smoothing effect on boundary pixels, itmay be effective to turn AIS off in use of CIP.

Subsequently, an intra prediction mode is determined (S1430).

The intra prediction mode is determined by a prediction unit (PU), inwhich an optimal prediction mode is determined in view of relationshipbetween required bit rate and amount of distortion.

For example, when rate distortion optimization (RDO) is on, a mode tominimize cost J=R+rD (R is bit rate, D is amount of distortion, and r isa Lagrange variable) may be selected. Here, thorough local decoding isneeded, in which case complexity may increase.

When RDO is off, a prediction mode to minimize a mean absolutedifference (MAD) by subjecting a prediction error to Hadamard transformmay be selected.

Table 1 illustrates a number of prediction modes with respect to a lumacomponent according to size of a prediction unit block.

TABLE 1 Block size Number of prediction modes 4 × 4 17 8 × 8 34 16 × 1634 32 × 32 34 64 × 64 3

FIG. 15 illustrates a prediction direction of an intra prediction mode.Referring to FIG. 15, a mode number 0 is a vertical mode in whichprediction is conducted in a vertical direction using a pixel value of aneighboring block. A mode number 1 is a horizontal mode in whichprediction is conducted in a horizontal direction using a pixel value ofa neighboring block. A mode number 2 is a DC mode in which a predictionblock is generated using an average pixel value of a current predictiontarget block, for example, a luma value in the case of luma pixels and achroma value in the case of chroma pixels. In other modes shown in FIG.15, prediction is conducted using pixel values of neighboring blocksdepending on corresponding angles.

In the DC mode, top prediction pixels and leftmost prediction pixels maybe filtered to enhance prediction efficiency. Here, intensity offiltering may become higher for a smaller block. The other internalpixels in the current prediction block may not be filtered.

Meanwhile, a planar mode to reflect directionality may be used insteadof the DC mode. In the planar mode, a Planar_flag value amonginformation transmitted from the encoder to the decoder is set to 1.When the planar mode is used, the DC mode is not used. Thus, when the DCmode is used instead of the planar mode, the Planar_flag value is set to0.

When the planar mode is employed, the same prediction methods asdescribed above in FIGS. 6 to 10 may be used. Here, the decoder mayperform an RDO operation described above so as to select optimal method.If necessary, two or more methods of the foregoing methods may be usedtogether. The encoder signals to the decoder information on which methodthe encoder selects among the prediction methods in the planar modeillustrated in FIGS. 6 to 10.

Regarding a reference pixel of a chroma component, unified directionalintra (UDI) of a luma block may be employed as it is in a mode number 4,which is referred to as a DM mode. In a mode number 0, a predictionblock is generated using linear relationship between a luma and achroma, which is referred to as a linear model (LM) mode. A mode number1 is a vertical mode, in which prediction is carried out in the verticaldirection, and corresponds to the mode number 0 of the luma. A modenumber 2 is a horizontal line, in which prediction is carried out in thehorizontal direction, and corresponds to the mode number 1 of the luma.A mode number 3 is a DC mode, in which a prediction block is generatedusing an average chroma value of a current prediction target block, andcorresponds to the mode number 2 of the luma.

Referring back to FIG. 14, the encoder encodes a prediction mode of thecurrent block (S1440). The encoder encodes a prediction mode for a lumacomponent block and a chroma component block of the current predictionblock. Here, since the prediction mode of the current prediction targetblock highly correlates with a prediction mode of a neighboring block,the current prediction target block is encoded using the prediction modeof the neighboring block, thereby reducing bit amount. Further, a mostprobable mode (MPM) of the current prediction target block isdetermined, and accordingly the prediction mode of the currentprediction target block may be encoded using the MPM.

Subsequently, a pixel value of the current prediction block and adifferential value by a pixel for the pixel value of the predictionblock are derived, thereby generating a residual signal (S1450).

The generated residual signal is transformed and encoded (S1460). Theresidual signal may be encoded using a transform kernel, wherein thetransform encoding kernel has a 2×2, 4×4, 8×8, 16×16, 32×32 or 64×64size.

A transform coefficient C is generated for the transform, which may be a2D block of transform coefficients. For example, for an n×n block, atransform coefficient may be calculated by Equation 20.

C(n,n)=T(n,n)×B(n,n)×T(n,n)^(T)  [Equation 20]

Here, C(n, n) is an n*n transform coefficient matrix, T(n, n) is an n*ntransform kernel matrix, and B(n, n) is an n*n matrix for a predictiontarget block.

When m=hN, n=2N and h=½, a transform coefficient C for an m*n or n*mdifferential block may be obtained by two methods. First, the m*n or n*mdifferential block is split into four m*m blocks and a transform kernelis applied to each block, thereby generating the transform coefficient.Alternatively, a transform kernel is applied to the m*n or n*mdifferential block, thereby generating the transform coefficient.

The encoder determines which to transmit among the residual signal andthe transform coefficient (S1470). For instance, when prediction isadequately performed, the residual signal may be transmitted as it iswithout transform encoding.

Determining which to transmit among the residual signal and thetransform coefficient may be carried out by RDO or the like. Costfunctions before and after transform encoding are compared to immunizecosts. When a signal type to transmit, that is, the residual signal ortransform coefficient, for the current prediction block is determined, atype of the transmitted signal is also signaled to the decoder.

Subsequently, the encoder scans the transform coefficient (S1480). Aquantized 2D block of transform coefficients may be changed into a 1Dvector of transform coefficients by scanning.

The scanned transform coefficient and the intra prediction mode areentropy-encoded (S1490). The encoded information is formed into acompressed bit stream, which may be transmitted or stored through anNAL.

FIG. 16 is a flowchart schematically illustrating an operation of thedecoder in the system according to the present invention.

Referring to FIG. 16, the decoder entropy-decodes a received bitstream(S1610). Here, a block type may be obtained from a variable lengthcoding (VLC) table, and a prediction mode of a current decoding targetblock may be derived. When the received bit stream may include sideinformation needed for decoding, such as information on a coding unit, aprediction unit and a transform unit, information on AIS filtering,information on limitation of a prediction mode count, information onunused prediction modes, information on rearrangement of predictionmodes, information on transform methods and information on scanningmethods, the side information is entropy-decoded along with the bitstream.

The decoded information may confirm whether a transmitted signal for thecurrent decoding target block is a residual signal or a transformcoefficient for a differential block. A residual signal or 1D vector oftransform coefficients for the differential block is obtained for thecurrent decoding target block.

Subsequently, the decoder generates a residual block (S1620).

The decoder inversely scans the entropy-decoded residual signal ortransform coefficient to generate a 2D block. Here, a residual block ismay be generated from the residual signal, and a 2D block of transformcoefficients may be generated from the transform coefficient.

The transform coefficients are dequantized. The dequantized transformcoefficients are inversely transformed, and the residual block for theresidual signal is generated via inverse transformation. Inversetransformation of an n*n block may be expressed by Equation 11.

The decoder generates reference pixels (S1630). Here, the decodergenerates the reference pixel by referring to information on whether AISfiltering is applied and on a used filter type signaled and transmittedby the encoder. Likewise in the encoding process, pixels in a rightmostvertical line of a left block already decoded and reconstructed andneighboring to the current decoding target block and pixels in a lowesthorizontal line of an upper block neighboring to the decoding targetblock are used for generating the reference pixel.

Meanwhile, when a CIP_flag value received by the decoder is set to 1,which means that the encoder uses CIP for a target picture, the decodergenerates the reference pixel accordingly. For example, only pixels ofneighboring blocks encoded in the intra prediction mode are used asreference pixels, whereas pixels of the neighboring blocks encoded inthe inter prediction mode are not used as reference pixels. In thiscase, as illustrated in FIG. 6, pixels (target prediction samples)corresponding to positions of the pixels of the neighboring blocksencoded in the inter prediction mode are generated as reference pixelsby interpolating the neighboring reference pixels encoded in the intraprediction mode, or the neighboring reference pixels encoded in theintra prediction mode may be copied and used as reference pixelscorresponding to positions of the pixels of the neighboring blocksencoded in the inter prediction mode.

For example, when intra prediction mode prediction pixels are present onboth right and left sides and upper and lower sides of a targetinter-prediction sample, the target prediction sample P_(T) located in ablock predicted in the inter prediction mode is derived by Equation 17.Further, when an intra-predicted sample is present only on either sideof the target prediction sample, the target prediction sample P_(T)located in a block location predicted in the inter prediction mode maybe derived by Equation 18. In Equation 17 and/or Equation 18, averagevalues of the corresponding intra prediction mode pixels may be used asP_(LB) or P_(RA) values. If there is no neighboring block predicted inthe intra prediction mode, a pixel value in the same position as in aprevious picture may be copied for use as a reference pixel value.

When the encoder employs AIS filtering, that is, when smoothing isapplied and thus AIS is on, the decoder also performs AIS filtering ingenerating the reference pixel according to the reference pixelgeneration method used by the encoder. The decoder may determine afilter coefficient based on filter type information among the receivedinformation. For example, when there are two filter coefficients [1, 2,1] or [1, 1, 4, 1, 1], a filter coefficient indicated in the filter typeinformation may be used among the two filter coefficients.

Next, a prediction block for the decoding target block is generatedusing the reference pixel and the entropy-decoded prediction mode of thecurrent decoding target block (S1640).

A process of generating the prediction block is the same as a process ofdetermining the prediction mode and generating the prediction block bythe encoder. When the prediction mode of the current block is a planarmode, a planner prediction method used to generate the prediction blockmay be identified by analyzing signaled information. Here, the decodermay generate the prediction block based on the identified informationaccording to a used mode among the planner modes illustrated in FIGS. 6to 10.

Next, a block reconstructed by adding, by a pixel, a pixel value of theprediction block and a pixel value of the differential block, that is, areconstructed block, is generated (S1670).

1-20. (canceled)
 21. A method of decoding a video signal with a decodingapparatus, the method comprising: obtaining a residual coefficient of acurrent block; deriving an inverse-quantized residual coefficient byperforming an inverse-quantization on the residual coefficient of thecurrent block; deriving a residual sample of the current block from theinverse-quantized residual coefficient of the current block; derivingreference samples for intra prediction of the current block based on atleast one of neighboring samples adjacent to the current block;obtaining a prediction sample of the current block by performing theintra prediction for a target prediction position of the current blockbased on a top reference sample having a same x-coordinate as the targetprediction position and a variation relating to the reference samples,wherein the variation relating to the reference samples isrepresentative of a difference value between a reconstructed value of atop-left reference sample whose a location is fixed regardless of acoordinate of the target prediction position and a reconstructed valueof a left reference sample whose a location is determined dependent onthe coordinate of the target prediction position, the left referencesample having a same y-coordinate as the target prediction position; andderiving a reconstruction sample of the current block by adding theprediction sample and the residual sample.
 22. The method of claim 21,wherein the prediction sample is included in a most left column of thecurrent block.